feat/add global SQS mode with single-queue-per-event and payload size control by nolramaf · Pull Request #1896 · EvolutionAPI/evolution-api

nolramaf · 2025-09-01T22:21:31Z

Introduce a global SQS configuration mode with unified queue provisioning and event routing, add large-payload handling via S3 fallback, and enhance queue management abstractions in SqsController.

New Features:

Introduce a global SQS mode to automatically provision and publish events to queues across all instances under a common prefix
Add S3 fallback storage for oversized SQS messages when payloads exceed MAX_PAYLOAD_SIZE and S3 storage is enabled

Enhancements:

Refactor SqsController to dynamically choose between global and per-instance event lists and queue naming based on configuration
Validate message payload size and switch dataType to ‘s3’ with automatic upload when exceeding SQS limits
Enable content-based deduplication for globally managed FIFO queues
Streamline queue listing, creation, and removal abstractions using a unified prefixName parameter

Build:

Extend environment configuration to include SQS_GLOBAL_ENABLED, SQS_GLOBAL_PREFIX_NAME, SQS_MAX_PAYLOAD_SIZE, and per-event flags
Expose HttpServer.NAME in config for richer message metadata

Summary by Sourcery

Implement a global SQS configuration mode with unified queue management and add large-payload handling via S3 fallback, while refactoring SqsController to simplify dynamic queue routing and message handling.

New Features:

Add global SQS mode for unified queue provisioning and event routing across instances
Introduce S3 fallback storage for oversize messages exceeding MAX_PAYLOAD_SIZE

Enhancements:

Refactor SqsController to select queues and naming dynamically based on global or per-instance mode
Validate message payload sizes and switch to S3 storage when exceeding SQS limits
Enable content-based deduplication on globally managed FIFO queues
Streamline queue listing, creation, and removal by using a unified prefix parameter

Build:

Extend configuration with SQS_GLOBAL_ENABLED, SQS_GLOBAL_PREFIX_NAME, MAX_PAYLOAD_SIZE, and per-event flags
Expose HttpServer.NAME in environment configuration

… control

sourcery-ai · 2025-09-01T22:21:37Z

Reviewer's Guide

This PR enhances SqsController by adding a global SQS mode with unified queue provisioning, refactoring event routing and queue management to use a common prefix, enforcing payload size validation with S3 fallback for oversized messages, updating FIFO queue deduplication, and extending environment configuration to support new SQS global flags and server metadata.

Sequence diagram for SQS message publishing with S3 fallback for large payloads

sequenceDiagram
    participant SqsController
    participant SQS
    participant S3Service
    actor Logger
    SqsController->SQS: sendMessage(message)
    alt Payload size <= MAX_PAYLOAD_SIZE
        SQS->SqsController: Message accepted
        SqsController->Logger: Log success
    else Payload size > MAX_PAYLOAD_SIZE & S3 enabled
        SqsController->S3Service: uploadFile(message)
        S3Service->SqsController: fileUrl
        SqsController->SQS: sendMessage({dataType: 's3', data: {fileUrl}})
        SQS->SqsController: Message accepted
        SqsController->Logger: Log success
    else Payload size > MAX_PAYLOAD_SIZE & S3 disabled
        SqsController->Logger: Log error (payload too large)
    end

Class diagram for updated SqsController queue management and event routing

classDiagram
    class SqsController {
      +init(): Promise<void>
      +set(instanceName: string, data: EventDto): Promise<any>
      +sendData(instanceName: string, event: string, data: any, ...): void
      +saveQueues(prefixName: string, events: string[], enable: boolean)
      +listQueues(prefixName: string)
      +removeQueuesByInstance(prefixName: string)
      -sqs: SQS
      -logger
      -monitor
      -prisma
      -configService
    }
    SqsController <|-- EventController
    SqsController --> SQS
    SqsController --> S3Service
    SqsController --> ConfigService
    SqsController --> Logger

File-Level Changes

Change	Details	Files
Async init and global queue provisioning	Converted init() to async and removed redundant Promise wrapper Loaded global SQS config and invoked saveQueues with prefix and events when GLOBAL_ENABLED is true	`src/api/integrations/event/sqs/sqs.controller.ts`
Unified event filtering and queue naming for global and per-instance modes	Merged global and per-instance event list selection into a single logic branch Determined prefixName based on GLOBAL_ENABLED flag or instanceName Constructed queueName and queueUrl dynamically using prefixName	`src/api/integrations/event/sqs/sqs.controller.ts`
Validate message size and fallback to S3 for oversized payloads	Measured JSON payload size against MAX_PAYLOAD_SIZE Uploaded oversized payloads to S3 when S3.ENABLE is true Updated message.dataType to 's3' and replaced data with fileUrl on fallback	`src/api/integrations/event/sqs/sqs.controller.ts`
Enhanced FIFO queue deduplication logic	Enabled ContentBasedDeduplication for globally managed FIFO queues Conditionally added MessageDeduplicationId only when global mode is disabled	`src/api/integrations/event/sqs/sqs.controller.ts`
Refactor queue management methods to use common prefixName	Renamed listQueuesByInstance to listQueues and accept prefixName Updated saveQueues and removeQueuesByInstance to generalize on prefixName Replaced instanceName-based prefixes and adjusted log messages accordingly	`src/api/integrations/event/sqs/sqs.controller.ts`
Extend environment configuration with global SQS flags and server metadata	Added SQS_GLOBAL_ENABLED, SQS_GLOBAL_PREFIX_NAME, SQS_MAX_PAYLOAD_SIZE, and EVENTS map to env.config Exposed HttpServer.NAME for richer message metadata Updated ConfigService to parse and expose all new variables	`src/config/env.config.ts`

Tips and commands

Interacting with Sourcery

Trigger a new review: Comment @sourcery-ai review on the pull request.
Continue discussions: Reply directly to Sourcery's review comments.
Generate a GitHub issue from a review comment: Ask Sourcery to create an
issue from a review comment by replying to it. You can also reply to a
review comment with @sourcery-ai issue to create an issue from it.
Generate a pull request title: Write @sourcery-ai anywhere in the pull
request title to generate a title at any time. You can also comment
@sourcery-ai title on the pull request to (re-)generate the title at any time.
Generate a pull request summary: Write @sourcery-ai summary anywhere in
the pull request body to generate a PR summary at any time exactly where you
want it. You can also comment @sourcery-ai summary on the pull request to
(re-)generate the summary at any time.
Generate reviewer's guide: Comment @sourcery-ai guide on the pull
request to (re-)generate the reviewer's guide at any time.
Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
pull request to resolve all Sourcery comments. Useful if you've already
addressed all the comments and don't want to see them anymore.
Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
request to dismiss all existing Sourcery reviews. Especially useful if you
want to start fresh with a new review - don't forget to comment
@sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

Enable or disable review features such as the Sourcery-generated pull request
summary, the reviewer's guide, and others.
Change the review language.
Add, remove or edit custom review instructions.
Adjust other review settings.

Getting Help

Contact our support team for questions or feedback.
Visit our documentation for detailed guides and information.
Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

sourcery-ai

Hey there - I've reviewed your changes and they look great!

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

DavidsonGomes · 2025-09-09T17:08:55Z

Please fix the lint with npm run lint

feat/add global SQS mode with single-queue-per-event and payload size…

cca929e

… control

sourcery-ai bot reviewed Sep 1, 2025

View reviewed changes

DavidsonGomes changed the base branch from main to develop September 2, 2025 13:01

nolramaf added 5 commits September 2, 2025 17:44

feat/force to save all evolution events in a single SQS queue

23cd6d2

feat/force MessageGroupId

17120e9

feat/force MessageGroupId

edb4fa3

feat/change variable GROUP_UPDATE to GROUPS_UPDATE

025b183

feat/change variable messageGroupId

7486d22

DavidsonGomes merged commit d998baa into EvolutionAPI:develop Sep 17, 2025
1 check failed

This was referenced Jan 25, 2026

Não está funcionando opção para habilitar SQS de forma global #1622

Closed

Ter o SQS disponível de modo global #932

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat/add global SQS mode with single-queue-per-event and payload size control#1896

feat/add global SQS mode with single-queue-per-event and payload size control#1896
DavidsonGomes merged 6 commits intoEvolutionAPI:developfrom
nolramaf:feat/add-global-sqs-mode

nolramaf commented Sep 1, 2025 •

edited by sourcery-ai bot

Loading

Uh oh!

sourcery-ai bot commented Sep 1, 2025 •

edited

Loading

Interacting with Sourcery

Customizing Your Experience

Getting Help

Uh oh!

sourcery-ai bot left a comment

Uh oh!

DavidsonGomes commented Sep 9, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

nolramaf commented Sep 1, 2025 • edited by sourcery-ai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by Sourcery

Uh oh!

sourcery-ai bot commented Sep 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviewer's Guide

Sequence diagram for SQS message publishing with S3 fallback for large payloads

Class diagram for updated SqsController queue management and event routing

File-Level Changes

Interacting with Sourcery

Customizing Your Experience

Getting Help

Uh oh!

sourcery-ai bot left a comment

Choose a reason for hiding this comment

Uh oh!

DavidsonGomes commented Sep 9, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

nolramaf commented Sep 1, 2025 •

edited by sourcery-ai bot

Loading

sourcery-ai bot commented Sep 1, 2025 •

edited

Loading